EDA of star dataset using Pandas Profiling

Dataset statistics

Number of variables	11
Number of observations	1100
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	94.7 KiB
Average record size in memory	88.1 B

Variable types

NUM	9
CAT	2

Warnings

`Star ColorRGB 0-255` has a high cardinality: 379 distinct values	High cardinality
`Bolo MagMbol` is highly correlated with `Abs MagMv`	High correlation
`Abs MagMv` is highly correlated with `Bolo MagMbol`	High correlation
`RadiusRstar/Rsun` is highly correlated with `LuminosityLstar/Lsun`	High correlation
`LuminosityLstar/Lsun` is highly correlated with `RadiusRstar/Rsun`	High correlation
`df_index` has unique values	Unique
`StellarType` has unique values	Unique

Reproduction

Analysis started	2020-12-08 04:46:53.405469
Analysis finished	2020-12-08 04:47:08.420853
Duration	15.02 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

df_index
Real number (ℝ_≥0)

UNIQUE

Distinct	1100
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	618.7072727
Minimum	1
Maximum	1235
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	1
5-th percentile	61.95
Q1	309.75
median	618.5
Q3	928.25
95-th percentile	1175.05
Maximum	1235
Range	1234
Interquartile range (IQR)	618.5

Descriptive statistics

Standard deviation	357.3266268
Coefficient of variation (CV)	0.5775374601
Kurtosis	-1.200770317
Mean	618.7072727
Median Absolute Deviation (MAD)	309.5
Skewness	-0.0006035148592
Sum	680578
Variance	127682.3182
Monotocity	Strictly increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1235	1	0.1%
415	1	0.1%
408	1	0.1%
409	1	0.1%
410	1	0.1%
411	1	0.1%
412	1	0.1%
413	1	0.1%
416	1	0.1%
386	1	0.1%
Other values (1090)	1090	99.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1	0.1%
2	1	0.1%
3	1	0.1%
4	1	0.1%
5	1	0.1%

Value	Count	Frequency (%)
1235	1	0.1%
1234	1	0.1%
1233	1	0.1%
1232	1	0.1%
1231	1	0.1%

Abs MagMv
Real number (ℝ)

HIGH CORRELATION

Distinct	190
Distinct (%)	17.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-1.213636364
Minimum	-9.5
Maximum	19
Zeros	7
Zeros (%)	0.6%
Memory size	8.6 KiB

Quantile statistics

Minimum	-9.5
5-th percentile	-9.3
Q1	-6.5
median	-3.8
Q3	3.1
95-th percentile	14.2
Maximum	19
Range	28.5
Interquartile range (IQR)	9.6

Descriptive statistics

Standard deviation	7.149003981
Coefficient of variation (CV)	-5.890565078
Kurtosis	0.2212808114
Mean	-1.213636364
Median Absolute Deviation (MAD)	3.7
Skewness	1.054579862
Sum	-1335
Variance	51.10825792
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-9.3	36	3.3%
-5.5	34	3.1%
-7	28	2.5%
3.2	27	2.5%
-5.1	25	2.3%
-6.5	24	2.2%
-4.7	22	2.0%
-2.5	19	1.7%
3.3	18	1.6%
-5.4	17	1.5%
Other values (180)	850	77.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-9.5	7	0.6%
-9.4	13	1.2%
-9.3	36	3.3%
-9.2	15	1.4%
-9.1	15	1.4%

Value	Count	Frequency (%)
19	8	0.7%
18.3	4	0.4%
17.9	4	0.4%
17.5	3	0.3%
16.8	8	0.7%

Bolo CorrBC(Temp)
Real number (ℝ)

Distinct	136
Distinct (%)	12.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-2.465527273
Minimum	-8.38
Maximum	-0.08
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	-8.38
5-th percentile	-6.93
Q1	-3.86
median	-1.99
Q3	-0.62
95-th percentile	-0.1
Maximum	-0.08
Range	8.3
Interquartile range (IQR)	3.24

Descriptive statistics

Standard deviation	2.082540225
Coefficient of variation (CV)	-0.8446632281
Kurtosis	0.2721092652
Mean	-2.465527273
Median Absolute Deviation (MAD)	1.61
Skewness	-0.8700099815
Sum	-2712.08
Variance	4.336973789
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-1.65	33	3.0%
-4.01	30	2.7%
-0.09	27	2.5%
-3.15	24	2.2%
-4.58	24	2.2%
-0.08	21	1.9%
-8.38	20	1.8%
-0.1	19	1.7%
-4.8	15	1.4%
-4.16	15	1.4%
Other values (126)	872	79.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-8.38	20	1.8%
-8.3	12	1.1%
-7.55	6	0.5%
-7.02	9	0.8%
-6.93	15	1.4%

Value	Count	Frequency (%)
-0.08	21	1.9%
-0.09	27	2.5%
-0.1	19	1.7%
-0.11	11	1.0%
-0.12	10	0.9%

Bolo MagMbol
Real number (ℝ)

HIGH CORRELATION

Distinct	582
Distinct (%)	52.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-3.679163636
Minimum	-17.38
Maximum	15.18
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	-17.38
5-th percentile	-12.484
Q1	-9.655
median	-5.85
Q3	1.6175
95-th percentile	11.19
Maximum	15.18
Range	32.56
Interquartile range (IQR)	11.2725

Descriptive statistics

Standard deviation	7.531126422
Coefficient of variation (CV)	-2.046966965
Kurtosis	-0.4848662686
Mean	-3.679163636
Median Absolute Deviation (MAD)	4.575
Skewness	0.7422590012
Sum	-4047.08
Variance	56.71786518
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-11.08	12	1.1%
-9.91	9	0.8%
10.7	8	0.7%
-8.85	8	0.7%
-7.15	7	0.6%
-9.51	7	0.6%
6.19	7	0.6%
8.73	6	0.5%
12.65	6	0.5%
14.7	6	0.5%
Other values (572)	1024	93.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-17.38	4	0.4%
-16.03	3	0.3%
-15.33	1	0.1%
-15.28	4	0.4%
-14.86	3	0.3%

Value	Count	Frequency (%)
15.18	6	0.5%
14.7	6	0.5%
14.11	6	0.5%
13.5	6	0.5%
12.94	2	0.2%

Color IndexB-V
Real number (ℝ)

Distinct	126
Distinct (%)	11.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.7154363636
Minimum	-0.37
Maximum	2.4
Zeros	8
Zeros (%)	0.7%
Memory size	8.6 KiB

Quantile statistics

Minimum	-0.37
5-th percentile	-0.35
Q1	-0.27
median	0.72
Q3	1.43
95-th percentile	2.25
Maximum	2.4
Range	2.77
Interquartile range (IQR)	1.7

Descriptive statistics

Standard deviation	0.8980828284
Coefficient of variation (CV)	1.255293796
Kurtosis	-1.355364519
Mean	0.7154363636
Median Absolute Deviation (MAD)	0.865
Skewness	0.2216615765
Sum	786.98
Variance	0.8065527666
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-0.35	78	7.1%
-0.34	39	3.5%
-0.32	39	3.5%
-0.33	33	3.0%
1.49	27	2.5%
1.41	25	2.3%
-0.3	24	2.2%
-0.31	24	2.2%
2.4	20	1.8%
-0.06	16	1.5%
Other values (116)	775	70.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-0.37	6	0.5%
-0.35	78	7.1%
-0.34	39	3.5%
-0.33	33	3.0%
-0.32	39	3.5%

Value	Count	Frequency (%)
2.4	20	1.8%
2.39	12	1.1%
2.26	9	0.8%
2.25	15	1.4%
2.17	5	0.5%

LuminosityLstar/Lsun
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	639
Distinct (%)	58.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4913954.586
Minimum	6.73e-05
Maximum	711000000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	6.73e-05
5-th percentile	0.00266
Q1	17.9
median	17350
Q3	576750
95-th percentile	7838500
Maximum	711000000
Range	711000000
Interquartile range (IQR)	576732.1

Descriptive statistics

Standard deviation	44828400.87
Coefficient of variation (CV)	9.122673009
Kurtosis	223.4465663
Mean	4913954.586
Median Absolute Deviation (MAD)	17349.9994
Skewness	14.55034464
Sum	5405350045
Variance	2.009585525e+15
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2150000	9	0.8%
0.00415	8	0.7%
0.265	6	0.5%
0.000105	6	0.5%
0.000693	6	0.5%
0.00182	6	0.5%
6.73e-05	6	0.5%
0.000315	6	0.5%
6.91	6	0.5%
437000	6	0.5%
Other values (629)	1035	94.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
6.73e-05	6	0.5%
0.000105	6	0.5%
0.00018	6	0.5%
0.000315	6	0.5%
0.00053	2	0.2%

Value	Count	Frequency (%)
711000000	4	0.4%
205000000	3	0.3%
108000000	1	0.1%
103000000	4	0.4%
69600000	3	0.3%

MassMstar/Msun
Real number (ℝ_≥0)

Distinct	254
Distinct (%)	23.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	24.94218182
Minimum	0.1
Maximum	160
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	0.1
5-th percentile	0.2
Q1	2.175
median	7.6
Q3	15.8
95-th percentile	130
Maximum	160
Range	159.9
Interquartile range (IQR)	13.625

Descriptive statistics

Standard deviation	41.29636978
Coefficient of variation (CV)	1.655683937
Kurtosis	2.710068057
Mean	24.94218182
Median Absolute Deviation (MAD)	5.95
Skewness	2.007523769
Sum	27436.4
Variance	1705.390157
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.1	35	3.2%
0.2	27	2.5%
0.8	21	1.9%
0.7	21	1.9%
0.5	20	1.8%
0.3	18	1.6%
0.6	18	1.6%
0.9	17	1.5%
0.4	14	1.3%
2.1	14	1.3%
Other values (244)	895	81.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.1	35	3.2%
0.2	27	2.5%
0.3	18	1.6%
0.4	14	1.3%
0.5	20	1.8%

Value	Count	Frequency (%)
160	3	0.3%
159.7	3	0.3%
159.4	3	0.3%
159	3	0.3%
158.7	3	0.3%

RadiusRstar/Rsun
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	594
Distinct (%)	54.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2701.506317
Minimum	0.00696
Maximum	231000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	0.00696
5-th percentile	0.00944
Q1	3.0375
median	20.5
Q3	235.5
95-th percentile	7567
Maximum	231000
Range	230999.993
Interquartile range (IQR)	232.4625

Descriptive statistics

Standard deviation	16541.59291
Coefficient of variation (CV)	6.123099844
Kurtosis	136.6412544
Mean	2701.506317
Median Absolute Deviation (MAD)	20.117
Skewness	10.89935991
Sum	2971656.948
Variance	273624295.8
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.00864	12	1.1%
20.2	11	1.0%
0.553	9	0.8%
15	9	0.8%
14.9	9	0.8%
19.8	9	0.8%
20.5	7	0.6%
0.00696	6	0.5%
12.5	6	0.5%
14.2	6	0.5%
Other values (584)	1016	92.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.00696	6	0.5%
0.00864	12	1.1%
0.00887	6	0.5%
0.00889	6	0.5%
0.0089	6	0.5%

Value	Count	Frequency (%)
231000	4	0.4%
103000	3	0.3%
87900	4	0.4%
68200	1	0.1%
51000	3	0.3%

Star ColorRGB 0-255
Categorical

HIGH CARDINALITY

Distinct	379
Distinct (%)	34.5%
Missing	0
Missing (%)	0.0%
Memory size	8.6 KiB

144 166 255	24
148 170 255	21
146 168 255	21
255 157 000	21
153 174 255	15
Other values (374)	998

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
144 166 255	24	2.2%
148 170 255	21	1.9%
146 168 255	21	1.9%
255 157 000	21	1.9%
153 174 255	15	1.4%
155 176 255	15	1.4%
151 172 255	15	1.4%
157 177 255	12	1.1%
255 204 143	12	1.1%
157 178 255	12	1.1%
Other values (369)	932	84.7%

Frequencies of value counts

Unique

Unique	152 ?
Unique (%)	13.8%

Histogram of lengths of the category

Length

Max length	11
Median length	11
Mean length	11
Min length	11

StellarType
Categorical

UNIQUE

Distinct	1100
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Memory size	8.6 KiB

N5Ia0	1
WC2III	1
M4VI	1
B9II	1
G3VI	1
Other values (1095)	1095

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
N5Ia0	1	0.1%
WC2III	1	0.1%
M4VI	1	0.1%
B9II	1	0.1%
G3VI	1	0.1%
DO5	1	0.1%
M8Ia	1	0.1%
WC4Ia0	1	0.1%
G7Ia0	1	0.1%
A3VI	1	0.1%
Other values (1090)	1090	99.1%

Frequencies of value counts

Unique

Unique	1100 ?
Unique (%)	100.0%

Histogram of lengths of the category

Length

Max length	6
Median length	4
Mean length	4.209090909
Min length	3

TempK
Real number (ℝ_≥0)

Distinct	171
Distinct (%)	15.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	14385.17545
Minimum	1990
Maximum	100000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	1990
5-th percentile	2180
Q1	3640
median	5670
Q3	24380
95-th percentile	47600
Maximum	100000
Range	98010
Interquartile range (IQR)	20740

Descriptive statistics

Standard deviation	16380.70046
Coefficient of variation (CV)	1.138720936
Kurtosis	3.104995851
Mean	14385.17545
Median Absolute Deviation (MAD)	2879
Skewness	1.660563497
Sum	15823693
Variance	268327347.6
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3700	25	2.3%
30200	24	2.2%
50000	24	2.2%
1990	20	1.8%
36800	15	1.4%
41200	15	1.4%
45600	15	1.4%
4669	15	1.4%
2180	15	1.4%
34600	15	1.4%
Other values (161)	917	83.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1990	20	1.8%
2000	12	1.1%
2167	9	0.8%
2180	15	1.4%
2288	5	0.5%

Value	Count	Frequency (%)
100000	6	0.5%
50400	6	0.5%
50000	24	2.2%
47800	15	1.4%
47600	9	0.8%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

First rows

	df_index	Abs MagMv	Bolo CorrBC(Temp)	Bolo MagMbol	Color IndexB-V	LuminosityLstar/Lsun	MassMstar/Msun	RadiusRstar/Rsun	Star ColorRGB 0-255	StellarType	TempK
0	1	-9.5	-4.58	-14.08	-0.35	34100000.0	160.0	80.2	144 166 255	O0Ia0	50000.0
1	2	-6.7	-4.58	-11.28	-0.35	2590000.0	150.0	22.1	144 166 255	O0Ia	50000.0
2	3	-6.5	-4.58	-11.08	-0.35	2150000.0	140.0	20.2	144 166 255	O0Ib	50000.0
3	4	-6.5	-4.58	-11.08	-0.35	2150000.0	130.0	20.2	144 166 255	O0II	50000.0
4	5	-6.5	-4.58	-11.08	-0.35	2150000.0	120.0	20.2	144 166 255	O0III	50000.0
5	6	-6.0	-4.58	-10.58	-0.35	1360000.0	110.0	16.0	144 166 255	O0IV	50000.0
6	7	-5.9	-4.58	-10.48	-0.35	1240000.0	100.0	15.3	144 166 255	O0V	50000.0
7	8	-5.6	-4.58	-10.18	-0.35	940000.0	60.0	13.3	144 166 255	O0VI	50000.0
8	10	-9.4	-4.43	-13.83	-0.35	27100000.0	159.7	78.8	146 168 255	O1Ia0	47600.0
9	11	-6.7	-4.43	-11.13	-0.35	2250000.0	149.3	22.7	146 168 255	O1Ia	47600.0

Last rows

	df_index	Abs MagMv	Bolo CorrBC(Temp)	Bolo MagMbol	Color IndexB-V	LuminosityLstar/Lsun	MassMstar/Msun	RadiusRstar/Rsun	Star ColorRGB 0-255	StellarType	TempK
1090	1226	10.2	-7.55	2.65	-0.37	6.910000	1.1	0.00902	155 178 255	DZ0	100000.0
1091	1227	10.8	-4.61	6.19	-0.35	0.265000	0.9	0.00696	159 181 255	DZ1	50400.0
1092	1228	11.4	-2.67	8.73	-0.27	0.025500	0.8	0.00864	168 189 255	DZ2	25200.0
1093	1229	11.9	-1.60	10.30	-0.20	0.006020	0.7	0.00944	179 197 255	DZ3	16800.0
1094	1230	12.5	-0.90	11.60	-0.13	0.001820	0.6	0.00922	192 207 255	DZ4	12600.0
1095	1231	13.1	-0.45	12.65	-0.07	0.000693	0.5	0.00890	207 218 255	DZ5	10080.0
1096	1232	13.7	-0.20	13.50	0.09	0.000315	0.4	0.00864	224 230 255	DZ6	8400.0
1097	1233	14.2	-0.09	14.11	0.34	0.000180	0.3	0.00889	243 243 255	DZ7	7200.0
1098	1234	14.8	-0.10	14.70	0.55	0.000105	0.2	0.00887	255 247 245	DZ8	6300.0
1099	1235	15.4	-0.22	15.18	0.74	0.000067	0.1	0.00898	255 239 225	DZ9	5600.0

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows